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Abstract 

Open educational resources (OER) have been identified as having the potential to extend 
opportunities for learning to non-formal learners. However, little research has been conducted 
into the impact of OER on non-formal learners. This paper presents the results of a systematic 
survey of more than 3,000 users of open educational resources (OER). Data was collected 
between 2013 and 2014 on the demographics, attitudes and behaviours of users of three 
repositories. Questions included a particular focus on the behaviours of non-formal learners and 
the relationship between formal and non-formal study. Frequency analysis shows that there are 
marked differences in patterns of use, user profiles, attitudes towards OER, types of materials 
used and popularity of different subjects. The experience of using OER is fairly consistent across 
platforms in terms of satisfaction and impact on future behaviour. On the whole, non-formal 
learners surveyed were highly positive about their use of OER and believe they will continue to 
use them. With regards to this making formal study more likely some degree of polarization was 
observed: some believed formal study was now more likely, while others felt it made this less 
likely. On the whole, while non-formal learners are enthusiastic about using free and online 
resources, the language and concept of OER does not seem to be well understood in the groups 
surveyed. A range of findings relating to OER selection and use as well as differences between 
repositories are explored in the discussion. 

Abstract in Spanish 

A pesar del potencial de los recursos educativos abiertos (REA) a la hora de ampliar las 
oportunidades de aprendizaje, existe poca investigation sobre su impacto en el aprendizaje no 
formal. Este estudio presenta los resultados de una encuesta realizada entre 2013 y 2014 para 
recoger datos demograficos, de actitud y comportamiento de mas de 3.000 usuarios de tres 
diferentes repositorios de REA, con especial atencion a la relation entre el estudio formal y no 
formal. El analisis de frecuencias muestra marcadas diferencias en las pautas de uso, perfil del 
usuario, actitudes hacia los REA, tipos de recursos utilizados y popularidad de diferentes 
materias. Las distintas plataformas registran experiencias de uso de REA similares en lo que se 
refiere a satisfaction e impacto sobre comportamiento futuro. Aquellas personas que aprenden 
en un contexto no formal se muestran muy positivas acerca del uso de REA y manifiestan su 
intention de continuar usando estos recursos. La transition hacia el aprendizaje formal tras el uso 
no formal de REA se presenta, sin embargo, polarizada entre aquellos que perciben el estudio 
formal mas probable y los que lo perciben menos probable. En general, mientras que las personas 
que aprenden en un contexto no formal se muestran entusiastas del uso de recursos gratuitos en 
la red, se observa la falta de entendimiento del concepto de REA entre los diferentes grupos 
encuestados. La ultima section explora una serie de conclusiones entre las que se encuentran, 
entre otras, diferencias en la selection y uso de recursos en los repositorios estudiados. 
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Introduction 

According to the definition provided by Hewlett (n.d.), open educational resources are “teaching, 
learning, and research resources that reside in the public domain or have been released under an 
intellectual property license that permits their free use and re-purposing by others”. These can 
include whole courses of open content, textbooks, multimedia, software and any other materials 
which may be used to teach or support learning such as lesson plans and curricula. There remains 
some debate about what should qualify as an ‘open’ resource, with some definitions emphasizing 
open access to resources and others focusing on the affordances for revising and repurposing 
afforded by open licenses (see Creative Commons, 2013). However, any disagreements tend to be 
limited to the specific kinds of licenses for educational that are termed ‘open’ and whether they 
should permit specific forms of re-use (such as only allowing non-commercial re-use). 

Evaluating the effects that OER have on learning is problematic for several reasons. Once 
materials are published as OER they can be adapted and repurposed in accordance with their 
licensing restrictions. This is often done to improve or update a resource, or to make it more 
closely aligned to a specific educational need. But this possibility for adaptation can also make it 
hard to judge the efficacy of a particular resource, and makes longitudinal studies problematic. 
Furthermore, even when data can be collected comparing similar cohorts who have consistently 
used OER and non-OER it can be difficult to reliably attribute any change in performance to the 
openness of the resources used. 

Most research exploring the impact of open educational resources (OER) has focused on learners 
who are registered for a course of study at an educational institution where OER — often in the 
form of open textbooks — are used. A recent review of efficacy and perception studies (Hilton, 
2014) found only twelve instances of peer-reviewed empirical research where the focus was on 
OER used as primary learning materials. These studies have indicated a high level of student 
satisfaction with the quality of OER available to them. Students have also consistently reported — 
particularly in the case of open textbooks — that OER have helped them financially (Pitt, 2015). 
OER use has also been correlated with higher test scores and lower rates of attrition (Hilton & 
Laman, 2012) though it is more common to find that use of OER delivers equivalent student 
satisfaction and performance at massively reduced cost (Feldstein et al., 2012; Wiley et al., 2012). 

The use of open educational resources (OER) by learners who are not registered for a formal 
programme of study has been subject to increased attention in recent years, with a recognition 
that there is a need for greater understanding of how to leverage non-formal learning in support 
of formal learning and support the transition from the former to the latter (Latchem, 2014; 
McGreal et al., 2014). 

Sangra and Wheeler (2013) have suggested that non-formal learning should increasingly be 
viewed as a viable alternative to formal education, but recognize that more research is needed to 
establish effective forms of support. Miyazoe and Anderson (2013) argue that: 

“the availability of ever-growing amounts of OER and the consequent non-formal learning 
opportunities fuel this \opening ’ of the traditional education systems. These free and open 
opportunities for both interpersonal and student-content interaction create an interaction 
surplus that can be used to augment and enhance formal educational curricula and systems 

The emergence of non-formal learning networks, facilitated by social networking and MOOC 
platforms, are increasingly recognised as a possible catalyst for non-formal learning. Siemens 
(2005) and Downes (2007) have outlined the ways in which such networks can be understood to 
support non-formal learning through the prism of Connectivism, which diminishes the 
distinction between formal and non-formal learning. 
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(r Non-formal learning is a significant aspect of our learning experience. Lormal education no 
longer comprises the majority of our learning Learning now occurs in a variety of ways — 
through communities of practice> personal networks , and through completion of work-related 
tasks ... Learning is a continual process, lasting for a lifetime. Learning and work related 
activities are no longer separate. In many situations , they are the same. ” (Siemens , 2005) 

The relationship between non-formal learning and formal study is increasingly an area for interest 
for higher education providers. Most research into non-formal learning concentrates on either 
the relevance of open materials for educational institutions, or on recognition of non-formal 
learning for professional development. As a result, little is known about the ‘non-formal’ OER 
learners; their ways of using these resources; or their reasons for studying in this way. 

Miyazoe and Anderson (2013) call for more attention to be paid to the “changing role of formal 
education in an era of learning opportunity where online educational resources and opportunities 
are readily accessible and in many cases completely free of cost to the learner”. Advocacy for 
non-formal learning tends to be grounded in the role that open resources can play in supporting 
formal learners who are registered for a course of study at an institution. OER can be used in this 
way to supplement curriculum, brush up on an area of study, for research purposes, or to 
otherwise complement traditional learning activities (Schmid et al., 2015). Much attention has 
been paid to possibilities for recognizing and accrediting the learning that takes place through self 
directed study which makes use of OER (Yang, 2015). OER also comprise a strategy for 
exposing non-formal learners with access to university level content; affording them 
opportunities to become more confident with self directed study. Institutionally produced OER 
can thus be seen to have both pedagogical and business dimensions. Many universities contribute 
to MOOC on the strategic assumption that in the long run they will be a source of future 
registrants. 

Part of the reason that non-formal learning has risen to greater prominence is a growing 
consensus that a flexible approach to learning and assessment can contribute to economic 
development and a richer understanding of how people learn in situ. But finding adequate ways to 
assess the effectiveness of non-formal learning remains a challenge. Mozilla Foundation (n.d.) 
launched the ‘Open Badges’ initiative to provide a framework for providing recognition for non- 
formal learning achievements through non-proprietary digital badges, which once awarded can be 
presented as part of a portfolio of learning. Badges tend to focus on skill acquisition rather than 
retention of ‘academic’ knowledge (Goligoski, 2012). As Glover and Latif (2013, p.1398) note, 
badges have the potential to support both student retention (through increased motivation) and 
future employability (through a digital record of achievement, or a portfolio of badges which can 
be shared with others as evidence of learning). 

A more traditional approach to the recognition of prior or non-formal learning is to follow 
private study with an examination equivalent to those taken by formal students. Conrad et al. 
(2013, p.46) have argued that digital and openly licensed learning materials are well suited to form 
the basis of a course of private study that can run parallel to institutional presentation and yet 
“[t]he greatest barriers to participation in open assessment and accreditation practices are 
identified as the lack of availability of committed staff members to support such activities, and 
the potential costs of redeveloping courses as OER”. Institutions who wish to promote 
recognition of prior learning therefore need to adopt a coherent, holistic strategy. 

<( Within institutions , key factors for the success of open assessment and accreditation 
implementation appear to be a reliance on a strong base of support within the institution — 
both in terms of leadership and resources—and an existing culture of openness that includes 
policies and practices around the creation and use of OER [...] Policies that enable either 
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open access or recognition of prior learning via credit transfer or RPL are also important. ” 

(McGreal et al., 2014, p. 130) 

The advantage of harnessing this kind of non-formal learning could be significant. Cofer (2000) 
estimates that every hour spent in formal study inspires up to four hours of non-formal learning, 
but insists that non-formal learning should not be seen as subordinate to formal study but 
valuable in its own right. Similarly, Latchem (2014) argues that as much as 70-90% of lifelong 
learning results from non-formal learning and skill acquisition; and yet the majority of research 
focuses on institutional learning. 

Evidently, there remains much to be known about non-formal learning and how best to integrate 
it with existing systems of assessment and accreditation. This study aims to improve our 
understanding of this topography by providing information on those who use online repositories 
of OER for private study. It should be noted that our aim is not to identify or profile distinct 
learning styles (Kolb, 1984) for non-formal learners. Not only has this approach been criticized as 
lacking in empirical support (Coffield et al., 2004) but such a categorization would not be 
possible without prior work on how resources are used by non-formal learners; their learning 
objectives; and study techniques used. 

Methodology 

This research took place within the context of OER Research Hub project which ran from 2013- 
2015 (OERRH, 2013). OERRH provided a focus for research on OER, and was designed to 
describe how openness is making a difference to learning and teaching practices. The project 
operated an open collaboration model and worked with a range of projects, initiatives and 
organisations across four education sectors (K12, college, higher education and non-formal 
learning ). In addition to the survey based research presented here, the project collected data 
through interviews, focus groups, institutional visits and through research fellowship. The open 
collaboration model involved negotiating the parameters and design of the research according to 
the needs of the collaboration partner. In order to ensure that the areas of focus remained 
consistent throughout the project the research was guided by eleven key hypotheses about open 
education (de los Arcos et al., 2014). The main hypotheses investigated across the project focus 
on the impact of OER on learners and the ways in which open licensing affects sharing and use 
of resources: 

• Use of OER leads to improvement in student performance and satisfaction; 

• The open aspect of OER creates different usage and adoption patterns than other online 
resources. 

In addition, several hypotheses pertained directly to non-formal learning: 

• Non-formal learners use a variety of indicators when selecting OER; 

• Non-formal learners adopt a variety of techniques to compensate for the lack of formal 
support, which can be supported in open courses; 

• Open education acts as a bridge to formal education, and is complementary, not 
competitive, with it; 

• Non-formal means of assessment are motivators to learning with OER. 

The survey was designed by the research team to collect evidence relevant to the project 
hypotheses concerned with non-formal learning from repository users. 
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Three OER repositories were selected for the study: iTunes U (Apple, 2014), OpenLearn (2015) 
and Saylor Academy (2015). iTunes U makes use of Apple’s iTunes content delivery systems — 
primarily used for commercial purposes — to make educational content available for free. This 
often comes in the form of podcasts and videos, and includes OER from museums and other 
cultural institutions as well as from higher education. The Open University (UK) runs has the 
most popular educational channel on iTunes U, though most major OER providers are 
represented in some way. Institutions can set up their own channel although can’t necessarily 
upload content to Apple servers. There is a course builder facility which is designed to help users 
create a programme of study within the platform. iTunes U is restricted to users of proprietary 
software/hardware made by Apple. Simply put, learners who wish to access these materials must 
at some point buy into the Apple ecosystem — even if only minimally. The advantage of this for 
the student is the assurance that all the technology should work adequately together, though there 
is a case for arguing that this is a significant impediment to openness and accessibility. 

OpenLearn (2014) is The Open University (UK) collection of free online learning materials, 
many of which are taken directly from courses presented to fee-paying students at the University. 
These include games and interactive media, materials from both undergraduate and postgraduate 
programmes as well as complete certified short courses. More than 33 million unique visitors 
have visited the OpenLearn website (which receives about 3 million yearly visitors) (Perryman, 
Law & Law, 2015). 

OpenLearn does more to accommodate non-formal learners who may lack confidence, providing 
short introductions to subjects as well as “interactive games, videos, blogs, [and] podcasts” 
(OpenLearn, 2015). A newsletter can also be subscribed to in order to get information about new 
courses as they come online. Most of the content on OpenLearn is designed to be explored 
through a web browser, and includes some multimedia content. It is structured so as to 
encourage the site user to browse their way straight into the content that interests them rather 
than 'choosing a course’ from the onset. This 'magazine’ style may be intended to ameliorate the 
potentially intimidating nature of the content for learners who are inexperienced or lacking 
confidence. On the homepage for OpenLearn there are often links made between current events 
(e.g. local and European elections) and relevant course content. This seems to be designed to 
make best use of pre-existing user interests, current TV shows and topicality to encourage the 
personal inquiry of the learner as the start of a learning pathway. 

The third repository in this study is Saylor Academy. Pounded in 2008, its mission is to use 
technology to drive the cost of a college education to zero. Working towards this through 
curating and compiling open content, Saylor acts as an aggregator of existing content, compiling 
OER into complete courses that emulate or complement existing college courses. Where Saylor is 
unable to source adequate open content they commission academic consultants to write 
additional content to fill gaps in the curriculum. In addition to curating college- length courses 
Saylor also offer a range of featured pathways through several OR collections. These can offer 
coverage similar to an introductory level liberal arts course, or training in core professional 
competencies. 

Upon completing a course of study with Saylor students can be examined. Lor successfully 
completing a Saylor commissioned programme of study students can earn a 'Saylor Certificate of 
Completion’ which, while conveying no institutional credit, provides evidence of learning and 
may be recognised differently in the future. Hilton, Murphy and Ritter (2014) have provided an 
account of the pedagogical and organizational theories informing the provision of OER as well as 
describing how these have been applied in the case of Saylor Academy resources. They argue for 
improved institutional recognition of non-formal learning, noting anecdotal evidence that 
students have gained college credit after studying non-formally using Saylor materials and 
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subsequently passing the same invigilated exam as formal learners following a non-open 
curriculum. 

One thing to note about the different repositories of OER is that they tend to be characterized 
by being free at the point of delivery to learners but the different platforms exhibit different 
potentials for re-use, re-appropriation and modularization of learning materials. Though Weller 
(2014) has argued that we need to remain open about openness and the possibility that it can mean 
different things in different contexts, what qualifies as open in different contexts remains 
contested. For some, an open licence which legitimizes re-use and re-appropriation is a minimal 
requirement. In addition, we might expect that materials are natively formatted and licensed in 
such a way as to encourage re-use and re-purposing (iTunes U, for example, would not meet this 
requirement). 

Collecting data from non-formal learners presents particular challenges around sampling and 
validity because of difficulties with verification. A multi-dimensional analysis (including factor 
analysis and smallest space analysis) of research literature published in the fields of non-formal 
and informal learning concluded that “the familiar qualitative and quantitative tools already in the 
hands of researchers, such as interviewing, observing, surveying, etc. are valuable and appropriate 
for studying non-formal education” and that no new tools or approaches were required to 
produce meaningful results” (Cohen, 2007). 

Data Collection 

OER repositories who acted as OER Research Hub collaborators — iTunes U, OpenLearn, and 
Saylor — circulated an invitation to complete the survey to non-formal learners who used these 
sites between 2013 and 2014. 2299 usable responses were received from users of Saylor 
Academy, 725 from OpenLearn users and 110 from users of iTunes U. The total sample size was 
3127 (though not all respondents answered all questions and some questions were omitted from 
the iTunes U survey). Precise sample sizes for each question are provided in the reporting below. 
Data was collected through SurveyMonkey, refined and compiled before being analysed in SPSS 
and Excel. 

Responses 

The survey was answered by OER users from a wide range of countries, as shown in Figure 1. 
Most responses came from the USA (n = 862) or the UK (n = 473) though India (n = 117) 
Canada (n = 87) and Brazil (n = 84) also contributed significant amounts of data. Most countries 
were represented and people from every continent contributed information. Excluding Africa, 
only Bolivia, French Guyana, Greenland, Kyrgyzstan, Suriname, Turkmenistan, Uzbekistan, and 
Venezuela were not represented. Most countries recorded between 5 and 40 responses. 
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Figure 1. Geographical spread of survey responses 


Results 


A set of demographic questions across the surveys asked about age, gender, prior qualifications 
and employment status to facilitate profiling OER users. As Figure 2 shows, the repositories 
exhibit clear differences in the age profile of their users. 



i Saylor(n=2299) 

I iTunesU(n=103) 
i OpenLearn (n=725) 


0 % 


5% 


10% 15% 20% 25% 30% 35% 


Figure 2. Age profiles of repository users (N = 3127) 


Users of iTunes U showed a much lower average age profile with 71.8% of their users aged 
below 35. By contrast, OpenLearn users tended to be older, with 69% aged 35 or over and 
relatively few younger users. The pattern of user age profiles was closer to a standard deviation 
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for Saylor users (perhaps reflective of a larger sample size). Similar patterns of difference can be 
discerned when considering the gender of respondents. 



Figure 3. Gender profiles of repository users (N=3090) 


Figure 3 indicates that Saylor users were more evenly split between male and female (and had a 
greater proportion of users who identified as transgender) while iTunes U users were more 
slightly more likely to be male. OpenLearn showed the largest difference in the gender of their 
users, with female users outnumbering male users by approximately half. 

Non-formal learners were also asked about their highest academic qualification and their 
employment status to ascertain differences in patterns of users across repositories. As is perhaps 
to be expected given the age profiles associated with the different repositories (Figure 3) 
iTunes U users, generally younger, were most likely to report holding a school leaving 
qualification as their highest. But as Figure 3 shows, the general trend was that users of these 
platforms tended to report already holding a degree. 59% of Saylor users held at least an 
undergraduate degree. The proportion of university graduates for OpenLearn and iTunes U were 
46% and 38.2% respectively. 
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Postgraduate/Graduate School 
University degree 

Undergraduate/Bachelors University 
degree 

College diploma or certificate 

Vocational qualification (i.e. practical, 
trade-based) 

School leaving qualification (16-18 years) 

No formal qualification 


0% 5% 10% 15% 20% 25% 30% 35% 40% 


■ OpenLearn (n=719) 

■ iTunesU (n=104) 

■ Saylor (n=2215) 


Figure 4. Educational profiles of repository users (N = 3038) 


The age of non-formal users of OER also seems important when profiling employment status. 
Figure 5 illustrates the main differences in pattern. Saylor users were much more likely to be in 
full time employment (52%, n = 2300) while the majority of iTunes U users (43%, n = 110) were 
in full time education. iTunes U also had the highest proportion of users who volunteer part time 
(15%, n = 110). Approximately 40% of OpenLearn users were in full-time employment, and 
OpenLearn had the highest proportion of retired users (14%, n = 732). 


Retired 
Unwaged (disability) 
Unwaged (domestic) 
Unwaged (seeking work) 
Student (part time) 
Student (full time) 
Volunteer (part time) 
Volunteer (full time) 
Employed (part time) 
Employed (full time) 



■ OpenLearn (n=732) 

■ iTunesU(n=110) 

■ Saylor(n=2300) 


0% 10% 20% 30% 40% 50% 60% 


Figure 5. Employment profile of repository users (N = 3142) 


OER repository users were also asked about the ways that they have typically connected to the 
internet in the previous three months. Figure 6 summarizes this information. 
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■ Saylor(n=2300) 

■ iTunesU (n=110) 

■ OpenLearn (n=700) 



Figure 6. Internet access profiles of repository users (N = 3116) 


When taken in conjunction with the geographical diversity of the sample these patterns suggest 
that those accessing these online resources in developing countries are more likely to do so 
through a smartphone, tablet, or home broadband. A small proportion (less than 10%) of 
respondents accessed the internet through a dialup connection. iTunes U users were more likely 
to access materials through a tablet or while at an educational institution or community facility. 

The final part of background information collected concerned disability. For the combined 
sample of all three repositories, 10% of users disclosed a disability (n = 3160). OpenLearn had 
the highest proportion of users who identified as disabled (15.7%. n = 737) while Saylor reported 
the lowest proportion of disabled users (7.9%, n = 2298). The relatively high rate of declaration 
by OpenLearn students may reflect similarities to the approach taken to improving accessibility at 
The Open University through an inclusive framework which covers human and technical 
elements from course production through to evaluation (McAndrew, Farrow & Cooper, 2012). 
Figure 7 provides more information on the types of disability reported, and shows that long term 
or chronic illnesses and mobility impairments were more commonly reported by OpenLearn 
users while mental health problems more likely to be reported by iTunes U users. iTunes U users 
were also much more likely to declare a speech disability. It should be noted that some 
respondents declared more than one disability. 


European Journal of Open, Distance and e-Learning - Vol. 18 / No. 2 
ISSN 1027-5207 
© 2015 EDEN 


58 























Who are the Open Learners? A Comparative Study Profiling non-Formal Users of Open Educational Resources 

Robert Farrow et al. 


QJ 

O 

i— 

Q. 

> 


_Q 

03 


LongTerm/Chronic 

Mental 

Neurological 

Learning 




10% 


■ OpenLearn (n=737) 

■ iTunesU (n=102) 

■ Saylor(n=2298) 


Figure 7. Disability profiles of repository users (N = 3137) 


With respect to reasons for using OER outside of a formal educational context, by far the most 
common response was a personal interest in a subject. Professional development and study 
relating to work were also popular. Particularly with users of Saylor. OpenLearn and Saylor users 
were more likely to report using OER to support the development of study skills or a second 
language. iTunes users were more likely than others to seek out OER for the purpose of sharing 
with others. Generally, iTunes users reported fewer reasons for accessing OER. Responses to 
this question are summarized in Figure 8. 



■ Saylor(n=2077) ■ iTunesU (n=94) ■ OpenLearn (n=612) 


Figure 8. Reasons for using OER (N=2783) 


The next part of the survey asked about the subject areas where open resources are typically used 
by the cohort. The responses are shown in Figure 9, where the most striking pattern is that users 
of iTunes U reported using OER across a much wider range of subject areas than users of the 
other repositories. This could be taken to suggest that users of this platform are more inquisitive 
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and diverse in their approach to self-study, or the fact that older casual learners access a broader 
range of OER as they indulge their curiosity about several subjects. Alternatively, this trend may 
reflect elements of the podcast model which allows users to download several resources that they 
are interested in at once to listen to later. In general, there was quite an even spread of interest 
across all subject areas. 
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Figure 9. OER repository use by subject area (N=2356) 


Across the entire sample (n = 2356) the most popular subjects for OER were computing (31.1%, 
n = 732) and economics (30.4%, n = 716). Table 1 shows the most popular subject areas by 
platform. 

Table 1: Most popular subject areas across repositories (N = 2356) 


Repository Most popular 2 nd ranked subject 3 rd ranked subject 

subject 


Saylor (n = 1750) 

Economics 

Computing 

Mathematics 


(34.2%) 

(33.2%) 

(28.2%) 

OpenLearn 

Languages 

Sciences / Arts 

Computing 

(n = 496) 

(29.8%) 

(24.6%) 

(23.8%) 

iTunes U 

Psychology 

Sciences 

Mathematics / Literature History 

(n = 110) 

(60.9%) 

(57.3%) 

(40.0%) 
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Respondents were also asked about the type and format of the resources they access. Videos, 
ebooks and open textbooks were the most commonly used resources. On the Saylor Academy 
platform ebooks and open textbooks were more popular than video. Saylor users reported using 
less multimedia content, games and quizzes but more lectures. No non-formal learners in any 
group reported using open data sets or lesson plans in their studies. Figure 10 summarizes the 
responses to this question. 


100 % 

90% 



■ Saylor(n=1543) ■ iTuneslI (n=107) ■ OpenLearn (n=435) 


Figure 10. Types of OER used (N = 2085) 


Users of Saylor and OpenLearn were also asked about their own reasons for accessing OER. 
(This question was not part of the iTunes U survey.) Figure 11 summarizes their responses. The 
most popular reason across both platforms was the chance to study at no cost. Saylor users were 
marginally more likely to cite the flexible and online nature of the resources while OpenLearn 
users were more likely to state that the chance to try university level content was important to 
them. 
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Figure 11. Reasons for using OER (N = 3025) 


These learners were also asked whether they did more than simply consume OER as it is 
presented to them through the platform they most often use; i.e. did they involve themselves in 
the kinds of re-use behaviours that are encouraged and legitimized by open licences? A relatively 
small number elected to answer this question, which may indicate a lack of confidence with the 
terminology used. Figure 12 presents their responses. It shows that Saylor users were much more 
likely to report engaging in processes that support OER production and evaluation. Although the 
sample size for this question was smaller, it is remarkable that more than 80% of Saylor users 
who answered claim to have adapted open materials for their own purposes as non-formal 
learners. (iTunes U users were not asked this question.) This is an interesting and counterintuitive 
result, possibly affected by different understandings around the concept of adaptation. 


Reviewed quality of public OER 
Added OER to a repository 
Publication of OER on open licence 
Creation of OER 

Adaptation of OER 



0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 


■ OpenLearn (n=210) ■ Saylor(n=697) 


Figure 12. Behaviours relating to use/re-use/review of OER (N = 907) 
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Survey respondents were also asked about the other repositories of OER that they have used to 
assess the extent to which platforms are used exclusively or in conjunction with each other. The 
results show that YouTube is the most popular place to find open resources, with over 50% of 
each sample reporting that they used it to find OER. iTunes U and TED talks were also popular 
across the samples, as was Khan Academy (though less so with users of OpenLearn). 


Saylor users were much more likely than the other groups to be studying via MOOC platforms 
(41.7%, n = 751). Figure 13 presents patterns in OER repository use. 


■ Saylor(n=1802) ■ iTuneslI (n=104) ■ OpenLearn (n=554) 



Figure 13. Patterns of OER repository use (N = 2460) 


To further understand patterns of use, the survey also asked about the indicators that non-formal 
learners look for when selecting OER (Figure 14). (Note that iTunes U users were not asked this 
question.) 
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Detailed description of resource 
Resource previously used by formal students 
Being required for formal study 
Attractive presentation 
Relevance to particular need 
Successful previous use 
Personal recommendation 
Positive reviews or ratings 
Interactive/multimedia content 
Length/complexity of resource 
Open licensing for adaptation 
Creative Commons licensing 
Reputation of creator 
Clear learning outcomes 
Ease of download/access 
Recently updated/uploaded 
Evidence of interest from others 



0% 10% 20% 30% 40% 50% 60% 70% 80% 


■ OpenLearn (n=729) ■ Saylor(n=2246) 


Figure 14. Indicators used by non-formal learners to select OER (N = 2975) 


We see a striking similarity between the responses given across the two samples. The most 
important criterion (for approximately 70%) was relevance to a particular need; i.e. OER were 
sought to plug a specific gap in knowledge or skills. The reputation of the OER creator (whether 
an individual or repository), clarity around learning outcomes or objectives, and ease of download 
or access were all cited as important. Less important were reviews of OER or personal 
recommendations. Open or Creative Commons licensing allowing adaptation was only thought 
important by a minority of 12-16% which is interesting given the high level of adaptation rates 
reported elsewhere (see Figure 12). It may be that non-formal learners adapt OER without paying 
much attention to licensing permissions. 

The least important factors appear to be the attractiveness of the presentation of a resource and 
whether an OER has previously been used in an institutional context. In the case of the latter it 
should be noted that despite this, more than 50% of the samples thought that the reputation of 
the resource creator was important and this reputation is often closely associated with 
institutional affiliation. Both Saylor (20%) and OpenLearn (21%) users identified a resource being 
required for formal study as important, suggesting that some of these learners may be using OER 
in lieu of non-open materials required for their formal studies. 
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Figure 15. Challenges faced when using OER (N = 1669) 


It is also noteworthy that between 16% and 25% of each sample believed that lack of support 
from a teacher or tutor was a challenge to their learning with OER. This could be seen to reflect 
a general level of confidence around independent, non-formal study among these groups, but still 
identifies a significant proportion who feel that they would benefit from more support of this 
kind. 


OpenLearn and Saylor users were asked in more detail about the techniques they employ to 
structure their learning when they have no formal support. Figure 16 shows their responses. 


Use of digital note-taking applications 
Use of a study calendar/plan 
Use of additional resources such as CDs, books, video 
Use of a learningjournal/diary 
Informal study groups 
Writing study notes 
Writing or reading blogs 
Consulting and/or editing wikis 
Discussion via videochat 
Discussion via microblogging 
Discussion in person 
Discussion in online forums 
Discussion via social networks 

0% 20% 40% 60% 80% 

■ OpenLearn (n=398) ■ Saylor(n=1494) 

Figure 16. Support techniques used by non-formal learners (N = 1892) 

The general pattern suggests that most non-formal learners on these platforms are not using 
many of the techniques described. Saylor users were more likely to report using a calendar or 
journal to organize their study as well as using specialized note-taking software; no OpenLearn 
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user reported using either of these. Writing study notes was in general more common among 
Saylor users (60.2%) while OpenLearn users were more likely to use a blog to record and 
organize their learning. Significant proportions of both samples said that they discussed their 
learning with others either face-to-face, via online forums, on social networks, or, less commonly, 
through micro-blogging platforms like Twitter or Tumblr. 


Finally, the survey asked about the likely impact of OER use in terms of the future behaviour of 
the learners. Figures 17-19 summarise their responses (with values for ‘more likely’ behaviours 
highlighted). 


Recommend repository content to others 
Share repository materials with others 
Make use of repository materials for teaching 
Download more materials from the repository 
Do further research in the subject 
Take a paid-for course 
Take a free course / study OER 



0% 20% 40% 60% 80% 

I More likely ■Nochange Less likely ■ Don't know 


100 % 


Figure 17. Impact of OER use on future behaviour of Saylor users (n = 1858) 


1 



Recommend repository content to others 


Share repository materials with others 


Make use of repository materials for teaching 


Download more materials from the repository 


Do further research in the subject 


Take a paid-for course 


Take a free course / study OER 


20 % 


0 % 


60% 


80% 


100 % 


■ More likely ■ Nochange ■ Less likely ■Don't know 


Figure 18. Impact of OER use on future behaviour of iTunes users (n = 94) 
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■ More likely ■ No change ■ Less likely ■ Don't know 


Figure 19. Impact of OER use on future behaviour of OpenLearn users (n = 583) 

We can discern here a very similar pattern of satisfaction with the quality of open materials across 
the three samples. Between 69% and 74% would recommend the repository to others and 
approximately half would share them directly with others (though the figure was slightly lower 
for OpenLearn at 44%). High proportions across all samples indicated that they would download 
further materials from the repository and feel empowered to undertake further study in a related 
area. But this satisfaction with the learning experience may also translate into a polarization 
between those who find that non-formal use of OER has made it more likely that they will seek 
formal study, and those who feel that it has made it less likely; presumably because of satisfaction 
with the kind of learning materials that are on offer openly. In each case those who answered 
‘more likely 5 or less likely 5 to this were about equal in number. OpenLearn users reported slightly 
greater likelihood of going on to formal study; this is perhaps reflective of the structured 
pathways leading from OpenLearn to degree programme course credits. 

The vast majority of those surveyed said that they would continue to use OER in the future — 
84.8% across the three samples as a whole. A mere 0.4% of those who participated in this study 
said that using OER had made it less likely that they would use OER in future. 

Discussion 

This dataset provides us with a more detailed sense of how non-formal learners using 
OpenLearn, iTunes U and Saylor Academy materials select and use OER. We can distinguish 
some broad patterns in the user bases of the different repositories. With a majority of the sample 
below the age of 30, iTunes U users were much more likely to be younger and were mostly male. 
They are often in full time education and use OER on an informal basis outside of their formal 
studies. By contrast, Saylor Academy users are more likely to be in employment and already in 
possession of a degree. They tended to be middle aged and primarily motivated by professional 
development. OpenLearn users were more likely to be older, retired, and female, and had a 
higher proportion of users who were motivated mainly by personal interest (though 40% are in 
full time employment). 
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Data about prior qualifications supports the pattern often seen in MOOC research (Laurillard, 
2014; Emmanuel, 2013) where users of free resources are often already graduates of higher 
education institutions. This was particularly evident with users of Saylor materials, more than half 
of whom (59%) were already in possession of at least an undergraduate degree. This was not the 
case for iTunes U, which had a markedly younger user base. 

One standout finding is that users of OER across all platforms expressed a high degree of 
satisfaction with the OER that they had accessed. Attitudes toward OER were overwhelmingly 
positive across the sample as a whole. This strongly supports the idea that OER (at least for the 
repositories studied) were of sufficient quality and appropriate for the education level of users. 
However, as Figure 19 indicates, this satisfaction does not always translate to a willingness to 
study formally. In fact, people who have used OER tend to become somewhat polarized. On the 
one hand we have those who feel more inspired or more confident after non-formal use of OER; 
while on the other there appear to be those whose learning needs are entirely met by use of open 
resources. Table 2 summaries this polarization. 


Table 2: Impact of repository use on likelihood of future formal study (n = 2535) 


Repository 

More likely to study formally 

Less likely to study formally 

iTunes U(n = 94) 

23.4% 

25.5% 

Saylor Academy (n = 1858) 

19.8% 

19.9% 

OpenLearn (n = 583) 

31.4% 

13.9% 


The higher figure for OpenLearn may be explained by the existence of planned pathways 
between OpenLearn content and degree level content provided by The Open University (UK) 
which are intended to facilitate the transition from non-formal to formal study. 

Even as learners reported a high level of satisfaction with the materials provided by their 
respective repositories, many identified difficulty locating subject-specific resources of adequate 
quality to be a significant challenge (Figure 15). In light of this it is noteworthy that there is a 
certain amount of 'brand loyalty 5 apparent with most learners using a small number of 
repositories (Figure 13). This can in part be attributed to lack of clear understanding of the nature 
of OER and what qualifies as a repository. It seems significant that specialist repositories of OER 
like Merlot, Jorum, and Curriki were hardly visited at all by the non-formal learners who 
answered the survey. This suggests that learners who use free online digital materials rarely visit 
specialized OER repositories, perhaps because they are only aware of resources being provided 
online for free rather than the concept and language of openness and OER. Respondents 
typically indicated a lack of understanding about the nature of an OER repository: between 9% 
and 20% of each sample said that they did not use any OER repositories despite the fact that 
they had only been offered the chance to participate in the survey specifically because of their 
OER repository use. 

One standout finding (Figure 12) was the high reported level of adaptation of OER by non- 
formal learners (as high as 85% for Saylor Academy users but still over 20% for OpenLearn). 
This was not anticipated and may indicate an unclear question. The research team intended by 
adaptation to mean some sort of editing or remix of resources in order to meet a perceived 
learning need. Given these responses, it may have been interpreted as collecting and curating 
resources together; or as taking material from one country/level in order to study in a different 
context. Comments received in relation to this question were not insightful, but did suggest that 
most informal OER users are not confident enough to remix and revise OER: not least because 
of the lack of interest in open licensing (Figure 14). This is an area where follow-up qualitative 
research focused on illuminating understandings of 'adaptation 5 would be beneficial. 
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A lack of knowledge about the technicalities of OER could be taken to imply that non-formal 
learners may not have much sense of reflecting on their own activity. Yet at the same time clarity 
of learning objectives was one of the most important factors affecting OER selection, so learners 
do seem to care about some aspects of pedagogy. In relation to this, it is noteworthy that no 
more than 25% of each sample felt that lack of tutor support was a barrier to them learning using 
OER (Figure 15). This confidence could be reflective of a number of factors, including faith in 
the quality and presentation of OER; skills gained through prior study; or overconfidence in ones 
ability to study unsupported. Data received about the impact of OER use on future behaviour 
would favour the first interpretation. As Figures 17-19 show, most users are overwhelmingly 
positive about their experience of using OER across all three repositories. 

Limitations and future research 

It should be noted that those who responded to the survey invitations constitute a self-selecting, 
self-reporting sample. A further limitation of the data is that respondents were free to skip any 
question they did not wish to answer. This means that there are some gaps in the data provided. 
Furthermore, in discerning patterns between the different user groups it should be borne in mind 
that the sample sizes were quite different across the three platforms. This may be particularly 
pertinent for the relatively small sample size of iTunes U users. Future studies could apply more 
rigorous sampling methods in order to extrapolate to more general populations. The lack of prior 
research in this area means that even with these caveats the results are likely to be of interest to a 
range of stakeholders. Further analysis of this data set (Farrow et al., 2015) is encouraged. Such 
work could examine patterns of response according to country; employment status; area of study; 
prior study; and other variables. 

Two areas that would likely make a good focus for qualitative work are to better unpack what 
non-formal learners understand by phrases like 'adaptation’ and 'relevance’, since these appear to 
be key considerations in the selection and use of OER. This may also facilitate understanding of 
why some learners ultimately embark on formal study as a result of their use of online resources. 

Conclusion 

This paper has presented data collected from users of three prominent repositories of OER. It 
has shown that there are marked differences in patterns of use, user profiles, attitudes towards 
OER, types of materials used and popularity of different subjects. Data also indicate that the 
experience of using OER is fairly consistent across platforms in terms of satisfaction and impact 
on future behaviour. Overwhelmingly, non-formal learners are very positive about their 
experiences of using OER, with a huge majority stating that they are more likely to use OER in 
the future. Indeed, such is the level of apparent satisfaction with OER use that it makes a 
significant minority less likely to (re-)enter formal education at all. While users are enthused about 
using free and online resources, the language and concept of OER does not seem to be well 
understood in the groups surveyed. 

In relation to the research hypotheses of OER Research Hub, we can offer the following 
concluding comments in relation to this study. 


European Journal of Open, Distance and e-Learning - Vol. 18 / No. 2 
ISSN 1027-5207 
© 2015 EDEN 


69 



Who are the Open Learners? A Comparative Study Profiling non-Formal Users of Open Educational Resources 

Robert Farrow et al. 


Table 3: Summary of findings in relation to OER Research Hub research hypotheses 


Hypothesis _ 

Non-formal learners use a variety 
of indicators when selecting OER 


Non-formal learners adopt a 
variety of techniques to 
compensate for the lack of formal 
support, which can be supported 
in open courses 

Open education acts as a bridge 
to formal education, and is 
complementary, not competitive, 
with it 


Non-formal means of assessment 
are motivators to learning with 
OER 


Comment _ 

Many factors appear to be at play, including: detailed 
metadata; subject area and level of study; format; perceived 
relevance; reputation of repository or creator; evident 
learning outcomes; ease of access; and evidence of interest 
from others. Less important factors included open licensing 
and attractive presentation. 

Non-formal learners use a wide range of study techniques, 
with taking notes; reading and writing blogs; discussion both 
online and in person among the most commonly reported 
techniques. 

While it appears to be the case that for many learners OER 
use makes them feel more likely to embark on formal study 
there also appears to be a significant minority of non-formal 
learners who believe their learning needs are being met 
without the need for a more supported or formalized 
experience. 

Online assessments were not identified as a major 
motivating factor; though they may be more important in the 
context of MOOC where learning assessment and 
certification is more central. 
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